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(54) Preventing conflicts in distributed systems 

(57) A token-based protocol used to prevent con- 
flicts between copies of the same data items residing in 
different servers of a replicated distributed system is 
provided. The servers in the system with database rep- 
licas are provided with token pointers that are each 
associated with a token. Token pointers are used to 
track the current possessor of the token. Whenever a 
server receives or transfers a token, its corresponding 
token pointer is updated to reflect which server pos- 
sesses the token. The use of token pointers advanta- 
geously eliminates the need for global cooperation by 
the servers to regenerate a lost token. 
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Description 

FIELD OF INVENTION: 

The present invention relates to a distributed 5 
processing system and, more particularly, to preventing 
conflicts among copies of the same data item within the 
distributed system. 

BACKGROUND OF INVENTION: ™ 

In distributed systems, access to data items is often 
facilitated by database servers. A database server pro- 
vides storage of data items grouped into databases. An 
application running on a client station, such as a per- 15 
sonal computer, can access a data item by connecting 
to the database server that stores the corresponding 
database. A common scenario associated with distrib- 
uted systems is that some clients are geographically 
located a great distance away from the server. Access 20 
to the database by these clients is generally by remote 
connections, e.g., dial up connections. Remote connec- 
tions can incur long distance charges which result in 
added cost. Also, remote connections tend to increase 
system response time, thereby decreasing system per- 25 
fbrmance. To improve performance and reduce cost in 
distributed systems, copies or replicas of the database 
are stored on multiple servers, each advantageously 
located in proximity to various groups of clients. Such 
systems are known as replicated distributed systems. 30 

However, the existence of multiple copies of the 
same data item requires database management proto- 
cols to prevent conflicts between copies. "Conflicts", as 
used herein, refer to the scenario where two or more 
copies of the same data item each contain different 35 
modifications. One technique of preventing conflicts 
from occurring is to ensure that all changes to the same 
data item are serialized and always applied to the most 
current version. This involves assigning a unique token, 
as a permission to make changes, for all copies of the 40 
same data item. Thus, the number of tokens in the sys- 
tem is equal to the number of data items in a database 
replica. Unless the server possesses the token associ- 
ated with a specific data item, it cannot perform modifi- 
cations to that data item. 45 

A problem with token-based systems is that the 
token can be lost as a result of a server failure or com- 
munication delays during token transfer. Recovery of 
the lost token requires regeneration of the token by the 
system. Conventional token regeneration techniques, sc 
such as those described in S. Nishio, K. F. Li and E. G. 
Manning, "A Time-Out Based Resilient Token Transfer 
Algorithm for Mutual Exclusion in Computer Networks", 
Proceedings of the IEEE Int. Conf. on Distributed Com- 
puting Systems, 1989. pp. 386-393 (broadcast-based 5t 
regeneration), involve executing a distributed consen- 
sus protocol to identify the server that has the most cur- 
rent copy of the data item and to agree on which server 
regenerates the token. This necessitates numerous 



exchanges of broadcast messages among the servers, 
which increases message traffic in the network. Clearly, 
such protocols are impractical, particularly in low band- 
width networks (or where the network is operating at or 
near its bandwidth capacity) or where communication 
cost for connections to the network is high. In addition, 
broadcast regeneration protocols require global cooper- 
ation among the servers in the system, which means 
that all servers in the network must be simultaneously 
operational before the token can be regenerated. Thus, 
the unavailability of one or more servers creates prob- 
lems in regenerating the token. Also, these protocols 
force all servers on the system to be interrupted from 
their normal operations while the token is being regen- 
erated. From the above discussion, it is apparent that 
there is a need for an improved token-based protocol. 

SUMMARY OF INVENTION: 

In a replicated distributed system having copies of a 
database located on different servers, a token-based 
protocol for preventing conflicts between copies of the 
same data item residing on different servers is provided. 
As used herein, the term "servers" refers to servers with 
database replicas as well as to those servers on which 
conflicts are to be prevented. The protocol advanta- 
geously enables regeneration of a lost token without the 
need for global cooperation among the servers. This 
advantage is achieved by providing servers with token 
pointers to keep track of the server possessing the 
token. In one embodiment, a token pointer is associated 
with each token in the system. 

Each time a server receives or forwards a token to 
another server, it updates its token pointer associated 
with the token to identify the current possessor of that 
token. The protocol, in some instances, may not neces- 
sarily inform the server (server^ which requested the 
token as to which server actually possesses the token. 
For example, if the server (server b ) identified in the 
requesting server's token pointer has subsequently 
transferred the token to another server (server c ) other 
than the requesting server, then servers token pointer 
will no longer point to the server that actually possesses 
the token. In this case, server b requests the token from 
server c on behalf of server a . According to one embodi- 
ment, server c sends the token to server b which forwards 
it to server a , the original token requestor. Thus, a token 
pointer provides to the token requestor a nexus, directly 
or indirectly (via an intermediate server), to the server 
possessing corresponding token. 

The protocol involves at most two servers at any 
instant during a token request, i.e., the server issuing 
the token request (requestor) and the server receiving 
the token request (requestee). Due to this feature, there 
; is no need for broadcasting messages to the servers in 
the system to determine which server has the token. 
Furthermore, the protocol determines that a token is 
lost when the requestor or requestee thinks that the 
other has the token, but neither does. If the token is lost, 
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then it last existed in either the requestor or requestee. 
In accordance with one embodiment, the server which 
possesses the more current copy of the data item is the 
server which last had the token and is responsible to 
regenerate the token. As such, successful regeneration 
of a lost token need only to involve the requestor and 
requestee. Therefore, the protocol eliminates the need 
for global cooperation among the servers in the system. 

RRIEF DESCRIPTION OF DRAWINGS: 

FIG. 1 is a simplified illustration of a conventional 
replicated distributed system; 
FIG. 2 is an exemplary site within the distributed 
system of FIG. 1 ; 

Figs. 3a-e are examples of servers and the con- 
tents of its respective token pointer; 
FIG. 4 is a flow diagram depicting the steps obtain- 
ing a token; 

FIG. 5 is a flow diagram depicting the steps for 
GetToken routine; 

FIG. 6 is a flow diagram depicting the steps for 
TokenHandler routine; 

FIG. 7 is a flow diagram depicting the steps for 
ReleaseToken routine; and 
FIG. 8 is a flow diagram depicting the steps for 
RequestHandler routine. 

DETAILED n FSHRIPTlON ™= THE INVENTION: 



The invention relates to a token-based protocol 
which prevents conflicts. The protocol provides efficient 
regeneration of a token after it has been lost by eliminat- 
ing the need for lost global cooperation of all servers 
with a database replica. As previously discussed, con- 
ventional token-based protocols require both global par- 
ticipation by all the servers as well as several rounds of 
broadcast messages to regenerate the token after it has 
been lost. This can lead to increased cost and system 
downtime. 

FIG. 1 is a simplified illustration of a conventional 
replicated distributed system 100. Such systems are 
described in Berson, r.t.pnt/ftgrver Architecture, 
McGraw-Hill, Inc., 1992. herein incorporated by refer- 
ence for all purposes. Distributed system 100 com- 
prises sites 110, 120, 130, and 140 networked together 
by communication links 150a-150f. The physical imple- 
mentation of the communication links is not important. 
For example, the communication links may be a local 
area network (LAN) or a wide area network (WAN) inter- 
connecting sites at geographically dispersed locations. 
The sites may also be "loosely connected", i.e., con- 
nected through dial-up connections or wireless connec- 
tions such as satellite links. 

FIG. 2 is an exemplary site 200 of the distributed 
system. As shown, the site includes a plurality of clients 
240a-240c networked to a server 210 by a LAN, MAN, 
or WAN. The clients, for example, may be personal 
computers, workstations, or terminals. Other clients (not 



shown) may be connected via remote links. While the 
clients are shown to be associated with a particular site, 
it is understood that they may connect to other servers 
at other sites within the system using remote links, such 
as dial-up connections using modems, wireless connec- 
tions such as satellite links, or connections via the inter- 
net Furthermore, it is understood that clients may have 
similar or different configurations from each other, and 
other clients (not shown) may be included as desired. In 
10 addition, one or more clients may be located on the 
server Server 210 is, in some embodiments, a main- 
frame computer system, a workstation, or a personal 
computer that includes non-volatile storage device 225, 
such as magnetic disks or optical disks for data storage. 
15 The data being stored in the storage device 225 
includes data files which are organized into one or more 
groups called databases. Each individual data file in the 
databases is referred to as a data item. 

Referring back to FIG. 1. a copy or replica of the 
20 same database is stored in the memory of the servers 
located at sites 110, 120, 130 and 140. For sake of brev- 
ity servers 1 10, 120, and 130 refer to servers located at 
sites 1 10, 120, 130 and 140, respectively. Furthermore, 
the servers at each site may also be referred to as 
25 nodes. The clients at the various nodes provide users 
with an interface to communicate with servers to read 
and update data items. An "update" refers to an opera- 
tion which modif ies of one or more data items. When 
updates are made to one replica, these updates are 
so propagated to other servers during a replication session 
to make all replicas consistent with each other. Conven- 
tional update propagation techniques such as those 
described in Lotus Notes Release 3 Application Devel- 
opers Reference Manual, 1993, Lotus Development 
35 Corporation, Cambridge, MA., can be employed to 
maintain replica consistency. 

Typically, update propagation is scheduled between 
a source and recipient server. The source or recipient 
server identifies which copies of data items in the 
40 source replica are more current than their correspond- 
ing copies in the recipient replica. The source server 
then propagates the more recent copies or updates to 
the recipient server. To facilitate identification as to 
which copy of the same data items are more recent, ver- 
45 sion information such as version numbers described in 
D K Gifford, "Weighted Voting for Replicated Data , 
Proceedings of the 7th ACM SIGOPS Symposium on 
Operating Systems Principles, (1979), pp. 150-159, or 
other conventional techniques for identifying versions of 
so data items are used. Version information are associated 
with the data items. Each time a copy of the data item is 
modified, its corresponding version information is 
updated. By comparing version information of two cop- 
ies of the same data item, a determination as to which 
55 copy is more current can be made. Whenever a server 
acquires a more current copy of a data item from 
another server, it also acquires the version information 
associated with that copy of the data item 

In accordance with the protocol of the invention, 
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servers in the system having a database replica main- 
tain pointers that are associated with each token. These 
pointers are herein referred to as TokenHolders. Since 
the number of tokens is equal to the number of data 
items in a replica, the number of TokenHolders in each 
server is also equal to the number of data items. In 
order to simplify the description of the protocol, illustra- 
tive embodiments are described using one TokenHofder 
per server (i.e., 1 data item per database replica). How- 
ever, it will be understood that the protocol is applied on 
a per-data item basis. Also, to facilitate ease of discus- 
sion, a subscript variable (e.g.,) is used to denote any 
server in the system and its corresponding components. 

The pointers are used to keep track of the server 
possessing the token. In one embodiment, 
TokenHolden identifies either 1) to which server server 
forwarded the token or 2) itself in the case where server 
has the token. Server modifies TokenHolderj each time 
it transfers or receives the token. For example, after 
server 110 forwards the token to server 120, 
TokenHolderj 10 will contain information identifying 
server 120, such as server 120's server identification 
(SID) and TokenHofder 12 o wil1 contain its own SID. 

When a client connected to server-, sends a request 
to update, for example server's copy of data item x in 
serverj it determines whether or not it has the token 
associated with x. If server-, has the token, it services 
the update request. Otherwise, servers sends a token 
request message to the server (serverj identified in 
TokenHolden. If server a does not have the token, 
server a examines its TokenHolder a and forwards a 
token request message to server b (which is identified in 
TokenHoldeQ on behalf of the server^ This request 
chain continues until the server possessing the token is 
reached. Assuming that server b has the token, one 
embodiment of the protocol calls for server b sends the 
token to server a which then sends the token to server,. 
After the transfer of the token by each server, its respec- 
tive TokenHolder is appropriately updated. As a result, 
TokenHolder b will contain server's SID while 
TokenHolder a and TokenHolderj will contain servers 
SID. 

As apparent from the above description, 
TokenHolder h in some instances, may not necessarily 
inform server, who exactly is the possessor of the token. 
However, the pointers, at the very least, provide a nexus 
to the server that actually holds the token. This tech- 
nique of updating the pointers enables a requesting 
server to obtain the token from the possessor, either 
directly or indirectly via an intermediate server. As such, 
token requests are serviced without broadcasting mes- 
sages system wide and waiting for the token possessor 
to respond to the message. To avoid loss of information 
during, for example, server failures, TokenHolders may 
be stored in non-volatile memory of the servers. 

In the case where the token is lost, the request 
chain continues indefinitely because no server in the 
system has the token. Eventually, a situation occurs 
where the requester thinks that the requestee has the 



token and vice-versa, but neither does. At this point, the 
system is aware that the token is lost. Furthermore, the 
token last existed in either the requester and requestee. 
Since this is the case, regeneration of the token need 
; only involve these two servers. The server which pos- 
sesses the more recent version of x is the server which 
last had the token. As such, this server regenerates the 
token. Thus, the protocol advantageously regenerates 
the token without requiring global cooperation of all 
so servers in the system with database replicas. 

Figs. 3a-3e are illustrative examples showing the 
dynamics of TokenHolder in replicated distributed sys- 
tem 100. The arrows identify the server to which each 
server's TokenHolder points. During initialization of sys- 
15 tern 100, a server is designated to generate tokens for 
the data item in the database. At this time, it is not 
important which server generates the tokens since all 
data item replicas should be identical. Referring to FIG. 
3a, server 140 is chosen to generate the token associ- 
20 ated with data item x. As a result, the TokenHolder 
associated with x at server 1 10, 120, and 130 all point to 
server 140 while the TokenHolden 40 points to itself. 

Referring to FIG. 3b, server 1 10 requests the token 
from server 140. After server 140 releases the token to 
25 server 1 1 0, it updates its TokenHolden 40 to reflect that 
server 110 now has the token. Upon receipt of the 
token, server 110 updates its TokenHolden 10 to show 
that it has the token. However, the TokenHolders for 
servers 120 and 130 remain unchanged, pointing to 
30 server 1 40 as the possessor of the token even though it 
is not. As a result, any token request message originat- 
ing at either 120 or 130 is sent to server 140, which then 
relays the request on behalf of the requesting server to 
server 110. Unless the token is lost, the request chain 
35 eventually reaches the possessor of the token. 

In FIG. 3c, server 120 subsequently requests the 
token from server 140. Upon receiving the request, 
server 140 relays this request to server 110 which has 
the token. Server 140 releases the token to server 140 
40 from which it received the request. Server 140, upon 
receiving the token from server 110, releases it to 
server 120. The sequence of token transfers causes 
servers 110, 120 and 140 to update their respective 
TokenHolders as follows. Server 110's TokenHolden 10 
45 points to server 140; server 140's TokenHolder 140 
points to server 120; and server 120's TokenHolder 12 o 
points to itself. Since server 130 was not involved in the 
token transfer process, its TokenHolder i 30 remains 
unchanged. 

50 The token request by server 140 results in a 
request chain which includes server 1 40 and 110. How- 
ever, this request chain may be collapsed when neces- 
sary to include only server 110 if the update 
propagation between any two servers includes propa- 
55 gating the contents of TokenHolder. FIG. 3d illustrates 
an example of how the request chain is collapsed during 
update propagation. As shown, the contents of the 
TokenHolders are similar to those shown in FIG. 3c. As 
part of the scheduled update propagation between 
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source server 140 and recipient server 130, data item 
version information as well as its associated 
TokenHolder are propagated to the recipient server. 
The TokenHolder associated with the copy of the data 
item which has the most recent version information is 
the more current pointer. For example, if 
TokenHolder 40 is more current, its information is 
adopted by server 130 in its TokenHolder 130 during 
update propagation. As a result, server 130 now points 
to server 120 (dotted arrow). Thus, the request chain 
which previously included servers 110 (crossed out 
arrow) has been collapsed to point directly to server 
120. Likewise, an update propagation session between 
servers 1 10 and 140, as shown in FIG. 3e, collapses the 
request chain, which previously included servers 140 
and 120, to include only server 120. 

From Figs. 3a-3c, the server possessing the token 
releases the token to the previous server in the token 
request chain, regardless of whether or not it is the orig- 
inal token requester. As a result, the token reaches the 
original token requester by traveling back along the 
request path. Alternatively, if information identifying the 
original token requester is included in the token request 
message, the token can be forwarded directly to the 
original requester, thus eliminating the need for the 
token to reverse the request path in order to reach the 
original requesting server. After the transfer of the 
token, the TokenHolder of the server possessing the 
token' will contain the SID of the original token 
requester. However, a side effect of this technique is 
that under some situations, it results in request chains 
that is longer than if information identifying the original 
requester is not included in the request message. 

A token can become obsolete. An "obsolete token" 
is a token which the system cannot ensure that it is 
associated with the latest copy of a data item. This may 
create potential conflicts in copies of the data item. Sit- 
uations which may make a token obsolete include, for 
example, arrival of a token at the requesting server after 
an error has occurred. Such error may be a server fail- 
ure or a timeout error. A "timeout error" is def ined as an 
error which results when the time it takes for a requester 
to receive the token exceeds the amount of time allotted 
by the system. This usually happens when a token mes- 
sage experiences unusually long delays or when the 
token is lost. The timeout error thus prevents a server 
from waiting indefinitely for a token which may never 
arrive. 

To prevent an obsolete token from being inadvert- 
ently received by the requesting server, the protocol 
provides servers with a counter (tnumbet) associated 
with each token. The tnumber records the number of 
times that the respective token' had arrived at the 
respective server or the number of times that an error 
had resulted from a token request. The current value of 
. tnumber is included in the request message sent by the 
requester (server a ). In addition, the tnumber of the 
requestee (server b ) is included in the token message 
which is sent back to server a . When a token arrives at 



the server a , it compares the contents of its tnumber a 
with that of tnumber b . If tnumber a = tnumber b , then 
the token is not obsolete. Otherwise, server a discards 
the obsolete token. As such, the counter enables a 
5 server to detect and ignore an unexpected token mes- 
sage (i.e., obsolete) that can occur as a result of a 
server failure or error. The contents of the counters, in 
some embodiments, are stored in non-volatile memory 
to avoid being corrupted or lost during a server failure. 
10 Storage of token counters in non-volatile storage 
requires a disk access every time a token counter is 
modified. To avoid the overhead associated with disk 
accesses, a different structure for token counters is 
employed. In accordance with one embodiment, each 
is token counter comprises two components such as (age, 
counter). Age, which is stored in non-volatile storage, is 
incremented each time a server reboots, for example, 
from a server failure. Counter is stored in unstable stor- 
age and is incremented in the same manner as 
20 tnumber. Using the format, token counter, is defined to 
be greater than token counter 2 if and only if age, > agea 
and counter > counter 2 . As such, monotonicity of the 
token counters are maintained as required by the proto- 
col However, modifying or reading the counters does 
25 not require a disk access. 

In multi-processing environments, multiple proc- 
esses can execute simultaneously. Concurrent execu- 
tion of processes can cause potential conflicts between 
different copies of same data item in a replica. For 
30 example, client a and client^ which are connected to the 
same node, request independent updates to the same 
data item at the same time. Unless some controls are 
placed in the system to serialize the update requests, a 
conflict will exist between clients and clients copy of 
35 the data item. These conflicts are referred to as mtra- 
node conflicts. To avoid intra-node conflicts, conven- 
tional concurrency controls such as lock variables are 
provided for the servers. Concurrency controls and 
locks are described in J. Gray and A. Reuter, Transac- 
40 tion Processing: Concepts and Techniques, Morgan 
Kaufmann, San Mateo, California, 1991, herein incorpo- 
rated by reference for all purposes. 

Locks are associated with node resources, such as 
routines used to service token requests. Typically, two 
45 operations are associated with each lock variable, lock 
and unlock. To serialize parallel activities, only one proc- 
ess can successfully lock (or acquire) a lock vanable at 
any one time. If other processes desire to acquire that 
lock variable, their requests are placed in a queue. 
so Once the process which has acquired the lock vanable 
unlocks (or releases) it, a waiting process is chosen 
from the lock queue as the next process by which to 
acquire the lock variable.. 

In accordance with one embodiment, the servers 
55 are provided with two lock variables, requesting and 
token associated with each token The use of a 
requesting lock variable ensures that a request for a 
specific token from the same server is not issued unless 
the previous request for that token has been serviced. 
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This prevents requesting the token twice. This lock vari- 
able also prevents a token from being transferred to 
another server until it is released by the server possess- 
ing it. The token lock prevents concurrent accesses of a 
token by multiple processes. (This is a standard concur- 
rency control within the server.) 

Referring to FIG. 4, a flow diagram depicting how a 
server acquires a token in accordance with an illustra- 
tive embodiment of the invention is shown. The flow dia- 
gram is separated into sections 401, 402, and 403. 
Section 401 reflects actions executed by the server 
(server^ issuing the original token request, while sec- 
tion 402 reflects action executed by the server (server b ) 
receiving the request. Section 403 indicates actions 
executed by an additional server (server c ), if necessary, 
in the request chain. At step 410, server a receives an 
update request for a data item x. At step 420, in 
response to the update request, server executes a rou- 
tine (GetToken) to obtain the token. The GetToken rou- 
tine examines server a *s TokenHolder a associated with x 
to determine which server in the system possesses x' 
token. If server a already has the token, it proceeds to 
step 475. If GetToken determines that another server 
(server b ) possesses the token, it sends a token request 
message at step 430 to the server b . This request mes- 
sage includes server's ID, server's copy of x' version 
vector (wector a ), and tnumber. Thereafter, server a 
waits for the token to return. To prevent server a from 
waiting indefinitely for the token, such as when the 
token is lost, a timeout mechanism is used. At the expi- 
ration of a predefined timeout period GetToken informs 
server a that it is unable to retrieve the requested token. 

Upon successful return of the token, GetToken pro- 
ceeds to step 475. At step 475, after server a obtains the 
token, it executes a Tokenhfandler routine. 
TokenHandler checks the tnumber of the token to make 
sure that it is not obsolete by comparing tnumber with 
tnumber a . H the token is not obsolete, server a updates 
it. Otherwise the token is ignored, TokenHolder a to 
reflect that server a now has the token. At step 480, 
server a performs the clients update request and 
releases the token to a requesting server at step 490 by 
executing a ReleaseToken routine. The ReleaseToken 
routine identifies the requesting server and updates 
TokenHolder a to point to that server. 

When server b receives the token request message 
from server a , it can be in one of three states: 1) server b 
thinks server a has the token, 2) server b has the token, 
or 3) server b has forwarded the token to server c . In 
response to the token request message, server b exe- 
cutes a RequestHandler routine at step 440. The 
RequestHandler examines TokenHolder b to identify the 
state of server b . State 1 indicates a problem because 
neither server a nor server b has the token (both servers 
think the other has the token). This state typifies the 
scenario where a server which possesses the token 
experiences a failure or unusually long message delay 
For example. server a issues a token request to server b 
which successfully fulfills this request, i.e, server b sends 



the token to server a and updates its TokenHoider to 
reflect this action. However, the token message to 
server a was delayed long enough for servers request 
to timeout. The timeout causes server's tnumber a to be 
5 incremented. Then, even if server a receives the token, it 
would be ignored because its associated tnumber b is 
obsolete (tnumber a * tnumber b ) As a result, the token 
becomes lost during transfer. 

If RequestHandler determines that server b is in 
10 state 1, token regeneration is initiated. This includes 
comparing the wector b of server's copy of x to that of 
server a at step 445. If server's copy of x is more recent, 
then server b is assumed to be the last server which had 
the token. Server b then regenerates and forwards the 
15 token to server a . Token regeneration may be achieved 
by causing server b to imitate a token arrival by execut- 
ing the TokenHandler routine at step 450. 
RequestHandler then proceeds to steps 455 and 460 to 
retrieve and release the token by executing the 
20 GetToken and ReleaseToken routines. The token, 
along with its copy of x, since it is more current, is sent 
to server a at step 470. If, on the other hand, server's 
replica of x is more current, then server b assumes that 
server a was the last server that had the token, resulting 
25 in server b sending a message at step 453 to inform 
server a to regenerate the token. 

If server b is in either state 2 or 3, RequestHandler 
executes the GetToken routine at step 455, which either 
executes ReleaseToken routine at step 460 and trans- 
30 fers the token to server a (state 2) or sends a token 
request message at step 458 to server c (state 3). 
Server c receives the token request message and proc- 
esses it in a similar manner as server b at step 440. This 
process continues until the request chain reaches the 
35 server possessing the token. 

Referring to FIG. 5, a detailed flow diagram of the 
GetToken routine is shown. At steps 500 and 510, the 
requesting and token lock variables are set to their lock 
states. Setting requesting to the lock state prevents 
40 server a from serving multiple token requests for the 
same token concurrent. In addition, the token variable is 
leaked to avoid other processes from obtaining the 
token until the current process is finished updating the 
data item. At step 520, TokenHolder a is examined to 
45 identify which server in the system possesses the 
token. If server a already has the token, GetToken pro- 
ceeds to step 570. If TokenHoider a points to server b , 
GetToken sends a token request message to the 
server b at step 530. The message includes server's 
so SID, tnumber at information identifying the data item 
being updated and its' associated version vector. 

At step 550, GetToken then waits for the token to 
arrive or for a timeout to occur. If the token arrives 
before a timeout occurs, GetToken. at step 570 returns 
55 a success message to inform server a that the token has 
been successfully acquired. If a timeout occurs, 
GetToken increments tnumber by 1 and notifies server a 
that it failed to receive the token at step 590. An unsuc- 
cessful acquisition of the token causes GetToken to 
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relinquish system resources to another process by 
releasing the concurrency controls, i.e., unlocking the 
requesting lock variables. 

FIG. 6 is a flow diagram of the TokenHandler rou- 
tine in accordance with one embodiment of the inven- s 
tion. At step 600, server a executes the TokenHandler 
routine when a token message arrives from server b . 
The token lock variable at server a is locked at step 710. 
At step 620, TokenHandler validates the token by com- 
paring the token counter (tnumber) from server b with 10 
that of server a . The token is valid when both counters 
are equal. If the wector a token is valid, TokenHandler 
proceeds to step 650 and compares the wector a with 
the wector b . If the comparison indicates that server's 
copy is more current, server a adopts servers copy of x is 
at step 660. Server a then updates wector a to equal 
weclor b at step 670 and proceeds to step 680. If 
server's copy of the document is less current or identi- 
cal to server's copy, then server a does not need 
servers of x. As such TokenHandler proceeds to step 20 
680 At step 680, tnumber a is incremented to reflect the 
arrival of the token. At step 685, TokenHolder a is also 
updated to ref lect that server a now has the token. After 
TokenHolder is updated, token lock is unlocked at step 

690. 25 

FIG. 7 is a flow diagram depicting the 
ReleaseToken routine. At step 700. server b receives a 
token request message from another server a and exe- 
cutes the ReleaseToken routine. The token request 
contains SID a , tnumber a and wector a from server a . At 30 
step 710, the ReleaseToken determines whether the 
token request message originated from another server 
or from itself by comparing the SID a with its own SID. If 
they are equal, the request originated from itself and 
then proceeds to step 770. Otherwise, the request ong- ss 
inated from another server. At step 720, the 
TokenHolder is updated with SID a . Optionally, at step 
730, the updated contents of TokenHolder is written to 
non-volatile memory. As previously discussed, this pre- 
vents TokenHolder from being lost if a server failure 40 
occurs. 

At step 740, ReleaseToken compares the wector a 
with wector b . If the servers copy is more current than 
server's copy, server b sends the token message which 
includes its tnumber b and its copy of the data item along 45 
with the wector b information to server a . Otherwise, 
server b only sends the tnumber b and its wector b asso- 
ciated with its copy of the data item with the token mes- 
sage at step 750 because server a already has a more 
current copy of the data item. After the token message so 
is sent, the request lock is reset to the unlock state. 

FIG. 8 is a flow diagram depicting the 
RequestHandler routine. At step 800, server b receives 
a token request message from server a and executes the 
RequestHandler routine. The message includes SID a , ss 
tnumber a , and wector a . At step 810, the TokenHandler 
routine compares SID a with the contents of 
TokenHolder b . If the two values are not equal, 
TokenHandler proceeds to step 820, where it calls 



GetToken. However, if the values where it calls 
GetToken are equal, the system proceeds to step 860. 
This indicates that the token for x is lost and needs to be 
regenerated. At step 860, server a compares the wee- 
tor of its copy of x with wector b to determine which 
server had the token last. If wector b 's is more current 
than wector a , a RequestHandler calls TokenHandler at 
step 870 to initiate regeneration of the token and pro- 
ceeds to step 820. If servers copy of x is more current, 
server a sends a token message to server b at step 880. 
The token message includes a zero vector, which is a 
wector with all its components set to zero. This ensures 
that the data item associated with the zero vector is the 
least current. The token message and zero vector 
inform server a to regenerate the token since it was the 
last server to have the token. 

At step 820, server a calls GetToken routine to gen- 
erate the token. After the requested token returns suc- 
cessfully, at step 830 TokenHandler calls ReleaseToken 
routine which sends the token to server. However, if 
server b fails to retrieve the token, i.e.. timeout, 
TokenHandler sends a failure message to server a . 

While the invention has been particularly shown 
and described with reference to various embodiments, it 
will be recognized by those skilled in the art that modifi- 
cations and changes may be made to the present inven- 
tion without departing from the spirit and scope thereof. 
The scope of the invention should therefore be deter- 
mined not with reference to the above description, but 
instead should be determined with reference to the 
appended claims, along with their full scope of equiva- 
lents. 



Claims 

1 In a computer network with a database comprising 
a plurality of data items, the network comprises a 
plurality of servers corresponding to a plurality of 
database replicas, each comprising copies of the 
plurality of data items, a method for preventing con- 
flicts among the different copies of the same data 
item in the database replicas comprising the steps 
of: 

providing a plurality of tokens corresponding to 
the plurality of data items, wherein possession 
of a token by one of the plurality of servers 
(possessor) grants permission to the posses- 
sor to update a data item corresponding to the 
token in the server's database replica; 
providing a plurality of token pointers in the plu- 
rality of servers corresponding to the plurality of 
tokens; and 

updating the token pointers to provide a nexus 
to the possessor of the corresponding token. 

2 The method of claim 1 further comprising the step 
of servicing a request by a client connected to the 
server, referred to a requestor, to update one of the 
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plurality of data items in the requestor's replica, 
wherein the step comprises the steps of: 

retrieving token associated with the data item; 
and 5 
performing updates on the data item after the 
token has been retrieved. 

The method of claim 2 wherein the step of retrieving 
the token comprises the steps of: 10 

examining the requestor s token pointer corre- 
sponding to the token associated with the data 
item and if the requestor s token pointer identi- 
fies the requestor as the possessor, then it is 
already has the token; and 
sending a token request message to a reques- 
tee server if the requestor's token pointer iden- 
tifies the requestee as the possessor, where 
the requestee is not the same as the requestor. 20 

. The method of claim 3 wherein the step the token 
request comprises information identifying the 
requestor (SID) and version information associated 
with the data item. 25 

. The method of claim 4 wherein the version informa- 
tion associated with the data item is a version vec- 
tor. 

30 

The method of claim 5 further comprises the step of 
servicing the token request wherein the step com- 
prises: 

examining the requestee's to determine the 35 
possessor of the token; and 

i. releasing the token to the requestor if the 
requestee's token pointer identifies the 
requestee as the possessor, 40 

ii. forwarding an intermediate token 
request message on behalf of the reques- 
tor to a second requestee if the reques- 
tee's token pointer identifies the second 
requestee as the possessor, where the 45 
second requestee is not equal to the 
requestee, and wherein the intermediate 
token request message comprises the 
requestee's SID and the version vector of 
the requestee's copy of the data item, and 50 

iii. regenerating the token if the requestee's 
token pointer identifies the requestor as 
the possessor. 

7. The method of claim 6 where the step of releasing 55 
the token to the requestor comprises: 

updating the requestee's token holder to reflect 
that the requestor is the possessor; 



determining which copy of the data item is 
more current by comparing the version vector 
of the requestee's copy of the data item with 
the version vector of the requestor's copy of the 
data item; and 

sending a token message by the requestee to 
the requestor, wherein the token message 
comprises the version vector of the requestee's 
copy of the data item if the requestor's copy of 
the data item is more current and further com- 
prising the requestee's copy of the data item to 
the requestee's copy if the data item is more 
current. 

8. The method of claim 7 wherein the regenerating 
step comprises: 

comparing requestee's version vector with 
requestor's version vector to determine which 
copy of the data item is more current; 
sending a token regeneration message from 
the requestee to the requestor informing the 
requestor to regenerate the token if the reques- 
tee's copy of the data item is more current; and 
if the requestee's copy of the data item is more 
current, regenerating the token by the reques- 
tee and releasing the token to the requestor. 

9. The method of claim 8 wherein the token regenera- 
tion message comprises a zero version vector 
wherein the components of the zero version vector 
are set to zero. 

10. The method of claim 7 further comprising the step 
of handling the token message by the requestor 
comprising: 

comparing the version vector from the token 
message with the version vector of the reques- 
tor's copy of the data item to determine which 
copy of the data item is more current; 
if the requestee's copy of the data item is more 
current, then 

i. adopting the requestee's copy of the data 
item into the requestor s database replica, 

ii. updating the requestor's version vector 
associated with the data item to equal the 
version vector of the requestee's copy of 
the data item, and 

iii. updating the requestor's token pointer to 
reflect that the requestor is the possessor 
of the token; or 

if the requestor's copy of the data item is more 
current, then updating the requestor's token 
pointer to reflect that the requestor is the pos- 
sessor of the token. 
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1 1 . The method of claim 1 0 further comprises the steps 
of: 

providing a plurality of token numbers in the 
plurality of servers, the plurality of token num- 
bers associated with the plurality of tokens; and 
incrementing the requestor's token number in 
response to the requestor receiving the token. 

12. The method of claim 11 further comprises the step 
of: 

incrementing the requestor's token number 
after a token request timeout occurs. 

13. The method of claim 11 wherein the step further 
comprises the steps of : 

comparing the requestor's token number with 
the requestee's token number associated with 
the token; 

receiving the token if the requestor's and 
requestee's token numbers are equal ; and 
ignoring the token if the requestor's and 
requestee's token numbers are not equal. 

1 4. A distributed system having a database comprising 
a plurality of data items, the system comprises: 

a plurality of servers; 

a plurality of replicas of the database wherein 
the replicas have copies of the plurality of data 
items, the plurality of replicas corresponding to 
the plurality of servers; 
a plurality of tokens associated with the plural- 
ity of data items, wherein possession of a token 
granted by a server (possessor) grants the pos- 
sessor permission to update a data item corre- 
sponding to the token; 

a plurality of token pointers are provided in the 
plurality of servers, the plurality of token point- 
ers corresponding to the plurality of tokens, the 
plurality of token pointers are updated in the 
plurality of servers to provide a nexus to the 
possessor of the corresponding token. 
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